Edge Server Selection for Device-Specific Network Topologies

ABSTRACT

A server selects edge-server sets to deliver data to client devices. To rapidly find an optimal or near-optimal edge-server set, the server constructs a trellis having a number of states at least equal to the number of edge servers to be included in an edge-server set. Each state comprises a plurality of nodes, wherein each node corresponds to one of the plurality of candidate edge servers. A trellis-exploration algorithm selects the edge-server set by providing interconnects between each node of a first state to each of a plurality of nodes in a next state, and for each node in a state, selecting a path corresponding to a best performance metric that connects to a node in a previous state. Each performance metric comprises network topology information, which can include channel measurements from candidate edge servers. When the edge-server set is selected, the server distributes content to the edge servers, which is then transmitted to the client devices.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. pat. appl. Ser. No.15/152,028, filed May 11, 2016, now U.S. Pat. No. 10,021,175, which is aContinuation of U.S. pat. appl. Ser. No. 15/076,000, filed Mar. 21,2016, now U.S. Pat. No. 9,806,953, which is a Continuation of U.S. pat.appl. Ser. No. 13/647,686, filed Oct. 9, 2012, now U.S. Pat. No.9,325,805, which claims priority to Provisional Appl. No. 61/550,048,filed Oct. 21, 2011; and which is a Continuation-in-Part of U.S. pat.appl. Ser. No. 13/036,778, filed Feb. 28, 2011, which claims priority toProvisional Appl. No. 61/308,997, filed Mar. 1, 2010; aContinuation-in-Part of U.S. pat. appl. Ser. No. 13/036,171, filed Feb.28, 2011, which claims priority to Provisional Appl. No. 61/308,997,filed Mar. 1, 2010; and a Continuation-in-Part of U.S. pat. appl. Ser.No. 13/036,812, filed Feb. 28, 2011, which claims priority toProvisional Appl. No. 61/308,997, filed Mar. 1, 2010.

BACKGROUND 1. Field of the Invention

The present invention relates generally to content delivery networks(CDNs), and more particularly, to media distribution in wirelesscommunication networks.

2. Introduction

Limited storage of mobile devices is driving cloud services in whichdata and software are stored on the network. However, in wirelessnetworks, limited wireless bandwidth, variable reliability of thecommunication channels, and mobility of the client devices discouragesoff-site data storage and produces significant challenges to mediadistribution.

The trend in 4G cellular is to make wireless client devices part of thenetwork infrastructure by enabling them to communicate with each otherand cooperate. For example, Cooperative MIMO, cooperative diversity, andad-hoc peer-to-peer networking are being incorporated into the 4Gwireless standards. These technologies expand coverage, extend range,and greatly increase bandwidth with minimal cost to wireless serviceproviders. These technologies also provide a significant change to thenetwork topology, as the conventional server-client topology is overlaidwith some clients functioning as gateways and routers for other clients.As a result, the nature of media distribution needs to adapt to thesenew network topologies, especially when client devices generate contentor relay content to other client devices.

For conventional media distribution over the Internet, providers of webcontent and applications typically deliver content from multiple serversat diverse locations in order to sustain a good end-user experienceunder high traffic loads. In a wireless network, mobile client devicesmay function as part of the CDN infrastructure. In some cases, mobiledevices may be configured to function as “surrogate” origin servers(e.g., Edge Servers). In some cases, mobile client devices may generatecontent for localized data and software services, such as maps, weather,and traffic.

The extension of a CDN to a wireless ad-hoc network introduceschallenges, including, among others, how to guarantee fault-tolerance asthe location, availability, and link quality of each mobile terminal maybe continuously changing; how to control how requests from end-users aredistributed to each Edge Server; and how to guarantee high performancefor end-users as network conditions change.

Accordingly, content delivery mechanisms need to adapt to highlyvariable network topologies and operating conditions in order to providereliable media and data services. These and other needs in the field maybe addressed by aspects of the present invention.

SUMMARY

In accordance with an aspect of the present invention, methods, systems,and software are provided for routing media in a CDN. In wireless widearea networks (WWANs), aspects of the invention employ network layerprotocols that exploit special properties of the wireless physicallayer. In peer-to-peer networks, WWAN nodes may be configured to performedge server functionality, such as to reduce WWAN congestion.

A method of selecting an edge-server set from a plurality of nodescomprises determining the network topology state in a CDN. The networktopology state may comprise bit-rate estimates for communication linksbased on network congestion and channel quality. A plurality ofcandidate edge-server sets is selected, and a performance metric foreach set is calculated. At least one of the candidate edge server setsis selected based on the performance metric.

In one aspect of the invention, an iterative method is employed forselecting the candidate edge-server sets. In each iteration, a newcandidate set is generated by appending, deleting, or changing at leastone node in a previous set. The network topology state may be updatedfor the new set. A performance metric is calculated for the new set,comprising a benefit minus a cost associated with implementing the newset. At the completion of the iterations, a best set is determined.

In accordance with one aspect of the invention, the performance metricis calculated from a backpressure routing algorithm.

In accordance with another aspect of the invention, the iterativeprocess of selecting and evaluating candidate edge server sets employs atrellis exploration algorithm. A trellis may be constructed using anumber of candidate edge servers. A fitness function is derived from amathematical relationship quantifying costs and network performanceimprovements corresponding to each edge server. The best edge serversets correspond to paths having optimal path metrics (i.e., branchmetrics) derived from the fitness function. Multiple iterations throughthe trellis may be performed to refine the selection of edge servers.

In another aspect of the invention, a method for processing a request bya client for an object comprises determining if the requesting client isa wireless device in a wireless network; determining if another clienton the wireless network has the requested object; determining if therequesting client can communicatively couple to the other client; anddirecting the request to the other client. The method may be performedby an edge server, a parent server, an origin server, or another networkdevice configured to intercept requests. In some aspects, the method maybe performed by requesting client.

Additional features and advantages of the invention will be set forth inthe description which follows, and in part will be obvious from thedescription, or may be learned by practice of the invention. Thefeatures and advantages of the invention may be realized and obtained bymeans of the instruments and combinations particularly pointed out inthe appended claims. These and other features of the present inventionwill become more fully apparent from the following description andappended claims, or may be learned by the practice of the invention asset forth herein.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features of the invention can be obtained, a moreparticular description of the invention briefly described above will berendered by reference to specific aspects thereof, which are illustratedin the appended drawings. Understanding that these drawings depict onlytypical aspects of the invention and are not therefore to be consideredto be limiting of its scope, aspects of the invention will be describedand explained with additional specificity and detail through the use ofthe accompanying drawings.

FIG. 1 is a flow diagram depicting a method for routing requests fornetwork resources in a CDN;

FIG. 2 is a flow diagram depicting a method for selecting a set of edgeservers in accordance with an aspect of the invention;

FIG. 3 is a flow diagram that depicts a method for selecting edgeservers in accordance with another aspect of the invention;

FIG. 4 is a block diagram of a CDN in accordance with one aspect of theinvention;

FIG. 5 depicts a network configuration in which aspects of the inventionmay be implemented;

FIG. 6 depicts a method for determining a network topology state in awireless network in accordance with an aspect of the invention;

FIG. 7 depicts a method for determining a network topology state in awireless network in accordance with another aspect of the invention;

FIG. 8 depicts a state transition diagram according to one aspect of theinvention that employs a recursive solution to the problem of estimatingthe state sequence of a discrete-time finite-state Markov process; and

FIG. 9 is a flow diagram illustrating a method for selecting edgeservers in accordance with an aspect of the invention.

FIG. 10 is a flow diagram depicting a method for routing a request foran object by a requesting client in a wireless network.

DETAILED DESCRIPTION

Various aspects of the disclosure are described below. It should beapparent that the teachings herein may be embodied in a wide variety offorms and that any specific structure, function, or both being disclosedherein are merely representative. Based on the teachings herein oneskilled in the art should appreciate that an aspect disclosed herein maybe implemented independently of any other aspects and that two or moreof these aspects may be combined in various ways. For example, anapparatus may be implemented or a method may be practiced using anynumber of the aspects set forth herein. In addition, such an apparatusmay be implemented or such a method may be practiced using otherstructure, functionality, or structure and functionality in addition toor other than one or more of the aspects set forth herein.

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the invention. It should be understood, however, thatthe particular aspects shown and described herein are not intended tolimit the invention to any particular form, but rather, the invention isto cover all modifications, equivalents, and alternatives falling withinthe scope of the invention as defined by the claims.

In accordance with an aspect of the invention, FIG. 1 is a flow diagramdepicting a method for routing requests for network resources in a CDN.The CDN comprises a network of computers containing copies of dataplaced at various nodes (i.e. edge servers, which are also referred toas proxy caches, content delivery nodes, or repeaters) of the network.Data content types include web objects, downloadable objects (e.g.,media files, software, documents), applications, streaming media, anddatabase queries. Other data content types may also be cached anddistributed in the CDN. By caching data, a CDN can increase accessbandwidth and redundancy while reducing access latency. For example, thecapacity sum of strategically placed edge servers can exceed the networkbackbone capacity.

In FIG. 1, an origin server receives a client request for a particularresource 101. The request includes a resource identifier for theparticular resource. Sometimes the resource identifier includes anindication of the origin server. A mechanism known as a reflector, whichmay be co-located with the origin server, determines how to handle therequest from the client 102. For example, the reflector may decidewhether to reflect the request or to handle it locally. If the reflectordecides to handle the request locally, it forwards it to the originserver 113. Otherwise, the reflector determines a “best” repeater toprocess the request and forwards the request to the selected repeater103. If the request is reflected, the reflector sends a modifiedresource identifier to the client 104 that designates the selectedrepeater. The client requests the resource from the repeater designatedin the modified resource identifier, and the repeater responds to theclient's request by returning the requested resource to the client 105.If the repeater has a local copy of the resource, it returns that copy.Otherwise, it forwards the request to the origin server or anotherrepeater to obtain the resource. Optionally, the repeater may save alocal copy of the resource in order to serve subsequent requests.

Some aspects of the invention provide for determining which nodes aredesignated as edge servers. Strategically placed edge servers decreasethe load on interconnects, public peers, private peers and backbones,increasing network capacity and reducing delivery costs. Some aspects ofthe invention provide for determining which content is stored ondesignated edge servers. Further aspects of the invention provide fordetermining which of the edge servers handle a specific request. Insteadof loading all traffic on a backbone or peer link, a CDN can redirecttraffic to edge servers. In some aspects of the invention, a clientrequesting a resource may be directed to request the resource fromanother client. Thus, a client may perform at least some edge serverfunctions.

FIG. 2 is a flow diagram depicting a method for selecting a set of edgeservers in accordance with an aspect of the invention. The methodcomprises determining a network topology state 201 for a network ofnodes. A plurality of edge-server sets is selected 202 from the set ofnodes, and a performance metric for each of the edge-server sets iscalculated 203. At least one of the edge-server sets is selected basedon the best performance metric(s) 204.

In step 201, the network topology state may comprise bit-rate estimatesfor communication links based on network congestion, channel bandwidth,communication channel quality, and the like. The network topology statemay also comprise a topology of demand for resources across the network.

As used herein, a network topology is a mapping of the configuration ofphysical-layer and/or logical connections between nodes. Networktopology may include the signal topology of the network. By way ofexample, a network topology may comprise a mapping of communicationchannels between the nodes. The network topology indicates which nodescan communicate directly with each other. A graphical mapping of thenetwork topology results in a geometric shape that may be used todescribe the physical and/or logical topology of the network.

As used herein, the network topology state is a network topologycomprising additional information about data flow between the nodes,such as measured and/or estimated transmission rates, distances betweennodes, queue backlogs, latency, signal types, channel estimates, errorrates, congestion, link reliability estimates etc.

Since CDNs generally deliver content over TCP connections, thethroughput is affected by both latency and packet loss. In order tomitigate both of these effects, CDNs traditionally place servers closeto users. Typically, a closer source enables faster, more reliablecontent delivery. Although network distance is not the only factor thatleads to best performance, providing user demand in the network topologystate can facilitate selecting and evaluating the edge-server sets andbetter optimize resource delivery and network loads.

FIG. 3 is a flow diagram that depicts a method for selecting edgeservers in accordance with another aspect of the invention. A step ofdetermining a network topology state 201 for a network of nodes maycomprise estimating bit rates 311 for each link in the network. When apath in the network is congested or faulty, data packets sent over thatpath can get dropped or delayed. In some aspects of the invention,determining the network topology state 201 may comprise acquiringlink-delay information and/or requests for retransmission, and thenprocessing that information to estimate the bit rates 311 or achievesome other measure of link performance pertaining to the networktopology. In one aspect of the invention, channel modeling may beperformed in step 201 for at least some of the links (such as when alink comprises a wireless channel). Since power levels, channel coding,error-correction coding, and symbol constellations can be indicators ofchannel quality and bit rates, such signaling parameters may be employedin the step of determining the network topology state 201.

In one aspect of the invention, bit-rate estimation 311 may compriseestimating link quality 321, obtaining link bandwidth 322, and/ordetermining congestion at each node 323. In one aspect of the invention,determining congestion 323 comprises obtaining queue backlog informationat each node. In some aspects of the invention, step 201 comprisesgenerating a network topology that is essentially a map indicatingconnections between nodes. In a wireless network, nodes may listen tocommunications between each node to determine which nodes it couldpotentially link to. A node may broadcast its possible connections (andoptionally, link quality estimates of those connections) to facilitategenerating the network topology.

In a mobile wireless network, it may be necessary to frequently updatethe network topology. The step of estimating link quality 321 maycomprise measuring channel quality of a wireless channel between nodesin a wireless network. For example, a link quality estimate may compriseSNR, BER, packet error rate, or other direct and/or indirectmeasurements of at least one wireless channel. The step of obtaininglink bandwidth 322 may comprise obtaining link bandwidth(s) allocated bythe network and/or measured link bandwidth (e.g., bandwidths activelyemployed in the link).

Optionally, determining a demand topology for the network 312 mayprecede step 201.

Determining the network topology state 201 may be part of a process forproviding congestion load balancing, such as when multiple communicationpaths are available to a wireless terminal for network access. Thenetwork topology state may comprise estimated achievable bit rates foreach link between a given wireless terminal to each available accesspoint. In some aspects, the network topology state may comprise measuredbit rates. Thus, any mathematical function, such as fitness function,based on the network topology state may comprise at least one ofpredicted performance (e.g., bit rates) and measured performance betweennodes in the network.

In step 311, the bit rates may comprise a function of congestioninformation and signal quality information received from network nodes.The network topology state may then be used to select one or more edgeservers 202, such as to provide congestion load balancing.

When an edge server set is changed (e.g., a new edge server is added tothe edge server set), this typically changes the network configurationthat best serves the clients, as the best estimated achievable bit ratefor at least some of clients may be improved by reassigning edge serversto those clients. A request from a client for given content is directedto a “best” content delivery node, wherein “best” could mean that thecontent is delivered quickest to the client. For example, an edge servermay be selected if it can deliver content more quickly than an originserver. The edge server may be selected based on its geographicalproximity to the requesting client. However, the edge server with theclosest proximity does not guarantee the best performance, as networkconditions (e.g., congestion, link quality, link bandwidth) can impedethat nodes ability to serve content. Other performance metrics may beused to select the best content delivery node. For example, latency(such as may be determined by number of hops in a peer-to-peer networkor queue backlogs at the nodes), link bandwidth, link bit-error rates,and/or some regional or global measure of network performance may beemployed as a performance metric when assigning edge servers to serveparticular clients. In a wireless network, step 202 may compriseselecting one or more of the wireless network nodes to function as edgeservers.

The network topology state is continually changing due to networkconditions. In a mobile wireless network, the locations of the nodes andclients may change. Thus, in some aspects of the invention, determiningthe network topology state 201 may comprise an iterative process ofre-evaluating and/or updating the network topology state. When a new setof edge servers is selected 202, such as when a new edge server is addedto an existing set, a reassignment of edge servers to clients typicallychanges network congestion. Consequently, changes in network congestioncan lead to updates in assignments of edge servers to clients. Thus, aniterative process for determining the network topology state can beeffective in handling this feedback condition. A stable network topologystate may be produced after a predetermined number of iterations or whensome predetermined stability criterion is met.

In another aspect of the invention, determining the network topologystate 201 may comprise generating a statistical model of the networktopology state. The statistical model may comprise a time-average ofnetwork loads, user demands, queue backlogs, bit rates, channel quality,geographical distributions of users, and/or other link-performance data.In some aspects, the statistical model may account for temporalvariations in network conditions. For example, network loads andgeographical distributions of users typically vary with time and oftenexhibit predictable patterns that can be incorporated into thestatistical model.

In some aspects of the invention, determining the network topology state201 may comprise determining a device network topology 325 for each of aset of device types. Some devices run specific software that may need tobe updated. In one aspect, edge server selection 202 for a patch to ahandheld device's operating system may differ from the set of edgeservers storing an update to a laptop operating system due todifferences in the geographical distributions of handheld devices andlaptop devices. For example, handheld devices are most commonly operatedon streets and roads, whereas content requests from laptops are morecommon in commercial and residential areas.

In some aspects, different devices are capable of receiving differenttypes of media. For example, laptops are capable of receivinghigher-resolution video than smart phones. Thus, a device networktopology may be based on devices having a predetermined set ofcapabilities. The use and distribution of certain devices can depend ondifferent times of the day and vary from day to day. For example, mobiledevices are more densely distributed on metropolitan highways duringrush hour. The use of mobile devices during lunch hour and rush hours istypically higher as well. The type of content requested may vary bothtemporally and geographically. Thus, determining a network topologystate may comprise determining a demand topology. Determining a demandtopology 312 may comprise anticipating that certain types of contentwill be requested from certain geographical locations at certain times.For example, the network will receive a high number of requests fortraffic information in the vicinity of metropolitan highways during rushhours. Thus, determining the network topology state can lead to theselection of edge server sets for specific content.

A performance metric is determined 203 for each of the plurality ofprovisional edge-server sets. The request-routing mechanism in aconventional CDN allocates servers in the content deliveryinfrastructure to requesting clients in a way that, for web contentdelivery, minimizes a given client's response time and, for streamingmedia delivery, provides for the highest quality. Thus, the performancemetric may comprise different metrics depending on the type of contentbeing served.

Determining the performance metric 203 may also comprises routeselection. A routing service may predict a best path for a data transferbetween a source location (e.g., a content provider origin server) and atarget location (e.g., a CDN edge server) by analyzing some performancemetric common to a set of possible routes. Route selection may alsocomprise employing intermediate nodes for transferring data packets fromone node to another node. Intermediate node selection may be based onnetwork performance data collected over time. Route selection maycomprise selecting routes between edge servers (e.g., a CDN edge serverand/or nodes in a mobile wireless network performing edge serverfunctionality) and client devices.

The performance metric may comprise link information between given nodesbased on ping data (age, loss and latency). Step 203 then computes thebest routes. In update mode, step 203 may comprise continually pollingthe potential routes to rank their performance, and then use the bestroute to evaluate the associated performance metric.

Costs associated with each route and/or each edge server in the set maybe included in the performance metric. For example, the content provideris charged for the amount of storage required on the origin site tostore their files. The content provider purchases a fixed amount ofstorage on the edge servers that is used to store a limited number offiles. The content provider is also charged for the bandwidth used toload the files onto the origin site and the bandwidth used to transferfiles from the origin servers to the edge servers.

According to some aspects of the invention, the performance metric maydepend on various systemic factors. For example, the monetary cost forallocating additional bandwidth on a server may be less expensive if thebandwidth for the server is below the commit level. Thus, aspects of theinvention may comprise determining monetary costs 331 associated withcandidate edge servers prior to selecting the edge server sets 202.These monetary costs may be used in the evaluation of the performancemetric for each set 203.

Network loads typically vary with time of the day and day of the week.Thus, a step of determining which of the candidate edge servers havelower loads 332 during a specific time period may be performed prior toselecting the edge server sets 202. Estimated server loads may be usedto in the evaluation of the performance metric for each set 203.

The performance of each server may vary with time. Thus, a step ofdetermining server performance (e.g., bandwidth, latency, andavailability) 333 may be performed prior to selecting the edge serversets 202. In wireless networks, step 202 may comprise selecting one ormore wireless network nodes to provide edge server functionality, suchas caching frequently requested content and/or content that is pertinentto client requests in a node's geographical location. Selection of thenodes 202 may comprise providing for bit-rate estimates based on signalquality and current congestion levels.

In one aspect of the invention, congestion information is obtained inresponse to probes of each candidate edge server (referred to as activescanning), while in another aspect of the invention, beacon packetscontaining congestion information from each edge server are used toestimate the achievable bit rate (referred to as passive scanning). Thesignal-quality information may be used to estimate the bit error rate(BER) of data served by each candidate edge server based on the signalto noise ratio or signal strength of a signal received from each edgeserver. In one aspect, the packet error rate (PER) is calculated fromthe BER. The server performance may be used in the evaluation of theperformance metric for each set 203.

In accordance with some aspects of the invention, at least one of thesets of edge servers is selected 204 based on a balance of providercosts (e.g., bandwidth costs) and end-user satisfaction (e.g., networklatency and packet loss between the server and the client).

Following determination of a best set of edge servers 204, the selectedset is implemented in the network (not shown). Since changing the set ofedge servers changes congestion load balancing, the steps 201-204 may berepeated as part of an update process. Thus, the network topology forthe selected set and candidate edge server sets may be periodicallyupdated. Furthermore, determining network topology 201 may compriseupdating assignments of clients to edge servers as network loads change.

FIG. 4 is a block diagram of a CDN in accordance with one aspect of theinvention. A client 400 comprises a device field, which may include oneor more client devices, such as client devices 421 and 422communicatively coupled to the client 400 via a personal area network(PAN). The client 400 comprises a client-side metrics manager 425. Insome aspects of the invention, the metrics manager 425 may reside on oneor more of the client devices 421 and 422.

The metrics manager 425 collects data from the client devices 421 and422 and manages client interactions with the CDN. From the CDN'sperspective, the client 400 is a router to a sub-network because the PANis essentially hidden from the CDN. Thus, the metrics manager 425formulates requests for media that is configured and formatted for thespecific client device 400, 421, and/or 422 in the PAN that is selectedto present the media. The metrics manager 425 may optionally conveydevice information that is otherwise hidden from the CDN.

According to one aspect of the invention, the local segment of the CDNthat serves the client 400 may comprise a WLAN, a cellular network, asatellite network, or some other wireless network in which it isdesirable to conserve bandwidth and/or power resources. Thus, if adestination client device in the PAN (e.g., one of the client devices421 and 422) has lower bandwidth needs than client 400, bandwidthresources are not wasted by serving the client 400 at its typicalresource level. Rather, the metrics manager 425 requests a media streamat a lower bandwidth. In another aspect of the invention, at least oneof the client devices 421 or 422 is configurable for presenting mediacontent at a higher bandwidth than is capable of being presented by theclient device 400. The metrics manager 425 requests a media stream atthe higher bandwidth. The local segment of the CDN may adapt to therequest by allocating multiple channels or a high-bandwidth channel tothe client 400.

In one aspect of the invention, at least one of the client devices isserved by a wireless link, and the metrics manager 425 manages clientinteractions with the network based on wireless network performancedata. For example, the metrics manager 425 may request a media streambandwidth based on the quality of a communications link serving at leastone of the client devices 400-402. The link quality may be indicated bythe wireless network performance data. The link quality may be indicatedby other factors, such as the amount of data stored in a client device'sbuffer, or a low-bandwidth status warning from a device in response todetected pixilation or frame slowing of media presented in the client'sdisplay. The metrics manager 425 may be responsive to status and/orwarning indicators generated by the client devices 400, 421, and 422 foradapting (e.g., formulating) the requests for the media resources.

In one aspect of the invention, a client device may be served by morethan one network. For example, a cellular handset may also have WiFicapability. The metrics manager 425 may be configured for conveyingnetwork connectivity information to the CDN.

An edge server 430 serves one or more clients, such as clients 400-402.Clients 401 and 402 each comprise one or more client devices (not shown)and a client-side metrics manager (not shown). The edge server 430comprises a network node metrics manager 435 configured for collectingdata from one or more client-side metrics managers (e.g., metricsmanager 425) and brokering network resources from the CDN to the clients400-402. The edge server 430 and the clients 400-402 it serves are partof a client field 410. The client field 410 is managed by the metricsmanager 435 for distributing network resources (e.g., media services) tothe clients 400-402.

The data employed by the metrics manager 435 may comprise raw and/orprocessed client-side data. For example, the data may comprise a numberof requests for media resources, types of media resources requested,and/or bandwidths of the requested media resources. The data maycomprise aggregations of the client-side data, such as the totalbandwidth requested. The data may comprise indications of changingnetwork loads (e.g., the number of requests for increased/decreasedbandwidth) and/or changes in network performance (e.g., number ofacknowledgements, number of requests for retransmission, measuredlatency, packet error rate, etc.). The metrics manager 435 allocates thenetwork resources from the CDN to the clients 400-402 based, at least inpart, on the data collected from the client-side metrics managers 425.

According to one aspect of the invention, the network node metricsmanager 435 receives a request from one of the clients 400-402 in theclient field 410. The metrics manager 435 may perform a check todetermine if a cached copy of the requested object is available at theedge server 430. Where a cached copy of the requested object isavailable to the edge server 430 (e.g., in a data store coupled to theedge sever), the edge server 430 transmits the cached copy of the objectto the client. The method may further comprise transmitting the requestto a parent server or an origin server if the cached copy is notavailable at the edge server 430. Alternatively, the metrics manager 435(or node cloud metrics manager 445) may reassign the client field 410 toa different edge server.

A parent server 440 comprising a node cloud metrics manager 445 iscommunicatively coupled to a plurality of client fields, such as clientfields 410-412. The metrics manager 445 is configured for collectingdata from multiple network node metrics managers, such as network nodemetrics manager 435, for distributing network services between edgeservers 430. The parent server 440 and the client fields 410-412 itserves is denoted as a digi-node field 420. The digi-node field 420 ismanaged by the metrics manager 445 for distributing network services tothe edge servers, such as edge server 430.

The data employed by the metrics manager 445 may comprise raw and/orprocessed data from the network node metrics managers (such as metricsmanager 435). The data may comprise a number of requests for mediaresources, types of media resources requested, and/or bandwidths of therequested media resources. The data may comprise aggregations of theclient-side data and/or data from the network node metrics managers. Forexample, the data may comprise the total bandwidth requested, or theamount of bandwidth requested at each edge server. The data may compriseindications of changing network loads (e.g., the number of requests forincreased/decreased bandwidth) and/or changes in network performance(e.g., number of acknowledgements, number of requests forretransmission, measured latency, packet error rate, etc.). The metricsmanager 435 distributes the network resources of the CDN between thenetwork nodes (e.g., edge servers) based, at least in part, on the datacollected from the metrics managers 435.

The CDN shown and described with respect to FIG. 4 is configured forreducing access latency and network backbone congestion by storingcopies of popular resources on edge servers, such as edge servers thatare in close proximity to requesting users. However, wireless networksare often employed as the final segment of a CDN. In such systems, it isdesirable to optimize the use of wireless network resources, such asbandwidth and power. The introduction of peer-to-peer and cooperativenetworking technologies improves the management of such resources whenimplemented in accordance with aspects of the invention.

In accordance with certain aspects of the invention, in a CDN employingwireless communication links to client devices via a wireless wide areanetwork (WWAN), edge server functionality may be located much closer tothe clients. For example, frequently requested content may be stored onthe WWAN infrastructure, such as on servers operated by WWAN serviceproviders, and/or even on the client devices. Requests for selectedresources may be reflected to WWAN servers and/or clients. In someaspects, the WWAN service providers, or even other client devices withina requesting client's local area network may process the requests. Whenclients form cooperative groups via connections from local area networksand communicate with each other, such as in a peer-to-peer manner, WWANcongestion can be reduced by storing content on the clients.

FIG. 5 depicts a network configuration in which aspects of the inventionmay be implemented. While the system shown in FIG. 5 comprises a LongTerm Evolution (LTE) wireless network, other types of networks thatemploy Cooperative MIMO, mesh networking, and/or peer-to-peer ad-hoccommunications may employ aspects of the invention.

LTE wireless networks, also known as Evolved Universal Terrestrial RadioAccess (E-UTRA), are being standardized by the 3rd GenerationPartnership Project (3GPP) working groups. LTE is a standard forwireless communication of high-speed data based on GSM/EDGE andUMTS/HSPA network technologies. LTE Advanced is a next-generation mobilecommunication standard being standardized by 3GPP as a major enhancementto the LTE standard. LTE Advanced includes specifications for NetworkMIMO and Cooperative MIMO.

Network MIMO is a family of techniques whereby a client in a wirelesssystem is simultaneously served by multiple access points (e.g., basestations) within its radio communication range. By tightly coordinatingthe transmission and reception of signals at multiple access points,network MIMO effectively reduces inter-cell interference.

Cooperative MIMO is a distributed antenna array processing techniquetypically employed in wireless mesh networking or wireless ad-hocnetworking. In wireless ad-hoc networks, multiple transmit nodes maycommunicate with multiple receive nodes. To optimize the capacity ofad-hoc channels, MIMO techniques can be applied to multiple linksbetween the transmit and receive node clusters as if the antennas ineach cluster were physically connected. Contrasted to multiple antennasresiding on a single-user MIMO transceiver, cooperating nodes and theirantennas are located in a distributed manner. In order to optimize thecapacity of this type of network, techniques to manage distributed radioresources are essential. In aspects of the invention, resourcemanagement techniques for Cooperative MIMO may be further configured forcontent management, such as for load balancing and reducing congestionon WWAN links.

According to some aspects of the invention, distributed computing isemployed in a Cooperative-MIMO network to coordinate media distribution.The evolution of distributed processing on multiple computing cores andthe evolution of cooperation between mobile antenna systems enhancesopportunities to perform cooperative work, such as sharing informationstorage and data processing, on multiple cores owned by different users.Aspects of the invention provide for brokering of network resources,increasing the availability of information storage and processing foreach client device in exchange for helping others, and potentiallyimproving network efficiency (e.g., reducing network congestion andenabling quicker access to content).

The network shown in FIG. 5 comprises WLAN groups of WWAN clientdevices. For example, WLAN 531 comprises client devices 500-505, whichare configured for communicating with a WWAN. Communication paths 571and 573 denote WWAN connections with a WWAN comprising cellular basestations (or Node Bs, or base transceiver stations) 541 and 542. Basestations 541-543 comprise radio equipment connected to the WWAN thatcommunicates directly with the WWAN client devices 500-505 and 510-513.WLAN 532 includes client devices 510-513 and may optionally includedevices 504 and 505, which also reside in WLAN 531. Communication paths572 and 574 denote WWAN connections with the WWAN.

Radio Network Controllers (RNCs), such as RNCs 561 and 562, aregoverning elements in the UMTS radio access network that are responsiblefor controlling Node Bs. For example, RNC 561 performs radio resourcemanagement and mobility management functions for base stations 541 and542. RNC 562 performs radio resource management and mobility managementfunctions for base stations 543. The RNCs 561 and 562 connect to aServing GPRS Support Node (SGSN) 563 in the Packet Switched CoreNetwork. A Serving GPRS Support Node (SGSN) is responsible for thedelivery of data packets from and to the mobile stations within itsgeographical service area. Its tasks include packet routing andtransfer, mobility management (attach/detach and location management),logical link management, and authentication and charging functions. AGateway GPRS Support Node (GGSN) 564 is responsible for the interworkingbetween the GPRS network and external packet switched networks, such asthe Internet and X.25 networks.

From an external network's point of view, the GGSN 564 is a router to asub-network because the GGSN 564 hides the GPRS infrastructure from theexternal network. Thus, in a conventional CDN, an edge server 580 doesnot see the structure of the GPRS network. However, in order to improveGPRS network efficiency, such as reducing congestion on WWAN links,aspects of the invention provide for edge server functionality insidethe GPRS network.

In one aspect of the invention, a WWAN client device may function as anetwork controller for its respective WLAN. For example, client device500 is the network controller for WLAN 531, and client device 510 is thenetwork controller for WLAN 532. The network controllers 500 and 510organize the WWAN devices within their respective WLANs to communicatewith the WWAN. The WWAN client devices coordinate their WWAN processingfunctions, and each client device may access the Internet through theGPRS network. The RNC 561 may employ multiple base stations 541 and 542for communicating with each WWAN device, such as to suppress inter-cellinterference via phase-coherent coordination and joint spatial filteringbetween the base stations 541 and 542.

Some client devices may access the Internet via alternative networkconnections. For example, client devices 500 and 501 are communicativelycoupled to an access point 530 that may provide connectivity to theInternet. The network controller 500 may control data communications viaalternative network connections that are available to the WLAN 531.

In some aspects of the invention, a WWAN client device (such as device512) may comprise a personal area network (PAN) 533 with other devices551 and 552. The PAN 533 may provide a route to Internet connectivity,such as via access point 520. This network connectivity information maybe made available to other devices in the WLAN 532, and such alternativenetwork connections may be utilized by other client devices 510, 511,and 513 in the WLAN 532.

In some aspects of the invention, WWAN client devices in different WLANsmay communicate with each other via a different network than the WWAN.For example, one WLAN may communicate directly with another WLAN in amesh or peer-to-peer manner. In some aspects, information may becommunicated from one WLAN 531 to another 532 via access points 530 and520. In other aspects, WLAN 531 and WLAN 532 may communicate with eachother via cooperative beamforming.

In accordance with one aspect of the invention, FIG. 6 depicts a methodfor determining a network topology state 201 in a wireless network.Nodes in the wireless network snoop on MAC headers of packettransmissions 601 within radio range. Passive scanning, such as packetsnooping, enables the nodes to estimate their local topology 602. Forexample, a node listens to the MAC header to obtain the source addressand destination address of the packet. The node listens for theacknowledgement (ACK) from the destination address to determine if thedestination node is within communication range. If the ACK is overheard,then the snooping node concludes that the acknowledging node is part ofits clique topology. Otherwise, it is in a hidden-node topology withrespect to the destination node. The node may process acknowledgementsignals, such as measuring the signal strength, to estimate the linkquality or potential bit rate. Further information about the other nodesmay be obtained from the MAC headers. For example, the OUI and serialnumber may be compared to a database of OUIs and serial number ranges toidentify the manufacturer and device model for each node. Each nodebroadcasts its local topology 603 so that one or more nodes collectinglocal topology information 604 can construct a regional network topologystate in the wireless network.

FIG. 7 is a flow diagram depicting a method for determining a networktopology state 201 in a wireless network in accordance with anotheraspect of the invention. At the beginning of a timeslot, channelinformation and any necessary control information is broadcast 701 by abroadcast node in the wireless network. This information may betransmitted over a dedicated control channel, or it may be appended toheader of packets transmitted on previous timeslots. Nodes provide animmediate ACK/NACK feedback to the transmitter 702, which informs thetransmitter if the packet was successfully received. The absence of anACK signal is considered to be equivalent to a NACK, as this absenceindicates that the receiver node did not detect the transmission. Thebroadcast node accumulates all of the ACK responses 703 and thentransmits a final message that informs the successful receivers of othersuccessful receivers 704. This transmission may include instructions forfuture packet forwarding. In one aspect of the invention, the successfulreceivers may respond (not shown) by transmitting addresses and linkinformation of nodes that are hidden from the broadcast node.Information about the successful receivers (and optionally, any hiddenreceivers) may be used to generate a network topology for the wirelessnetwork. Simple estimates of channel quality may be made based on thenodes' feedback, or the nodes may return link-quality information.

In one aspect of the invention, the step of selecting edge-server sets202 in a wireless network comprises exploiting either or bothtransmission-side and receiver-side diversity made possible by theWireless Broadcast Advantage (WBA). An example of WBA is when a firstnode transmits a signal to a second node, and a third node receives thetransmitted information at no additional cost.

A transmission that might be overheard by multiple receiver nodes withinrange of the transmitter enables a multi-receiver diversity gain,wherein the probability of successful reception by at least one nodewithin the subset of receivers is often much greater than thecorresponding success probability of just one receiver alone.Cooperation between the second and third nodes for receiving thetransmission provides for receive-side diversity, which results in aperformance gain that can be used for a combination of reducing transmitpower and increasing data throughput. For example, some form of optimalcombining may be performed by the second and third nodes to achievereceiver-diversity gain.

Cooperation between the second and third nodes when rebroadcasting thetransmission produces transmit diversity, which can also provide aperformance gain. For this reason, the problem of finding the optimalpath between an edge server and each requesting client node may be amulti-stage decision-making problem, wherein at each stage, a set ofnodes may cooperate to relay the transmission to a selected node.

In one aspect of the invention, solutions to this multi-stagedecision-making problem may be estimated using an asymptotically optimaldecoding algorithm, such as a trellis-exploration algorithm similar tothe Viterbi algorithm. Such algorithms may be employed in steps 202,203, and/or 204. In one aspect, the minimum-energy cooperative route maybe viewed as a sequence of sets of cooperating nodes along with anappropriate allocation of transmission powers. The tradeoff is betweenspending more energy in each transmission slot to reach a larger set ofnodes, and the potential savings in energy in subsequent transmissionslots due to cooperation. Accordingly, selecting the candidate edgeservers 201 may comprise considering local and/or regional wirelessnetwork topologies that provide for cooperative routing, and estimatingpreliminary performance metrics for how well each edge server can serverequesting clients within its radio range.

In accordance with another aspect of the invention, WWAN nodes that haveaccess to the Internet via alternative networks may be selected as edgeservers, and requests for content by WWAN clients may be interceptedwithin the WWAN and redirected to one of the edge servers that canretrieve the requested content without loading the WWAN.

FIG. 10 is a flow diagram depicting a method for routing a request foran object by a requesting client in a wireless network. The request maybe processed by a conventional CDN edge server, parent server, or originserver. Alternatively, the request may be intercepted, such as by a WWANcontroller or by another client device. In some aspects, the requestingclient may perform at least one of the steps in the method.

Upon receiving the request, the receiver of the request determines ifthe requesting client is a wireless device residing on a wirelessnetwork 1001. The requesting client may simply identify itself as awireless device. If the receiver of the request is a wireless networkcontroller, then step 1001 may comprise authenticating the client'sidentity. If the receiver of the request is a server on the CDN, then apredetermined client identifier in the request may be cross-referencedwith a client database to determine if the client resides on a wirelessnetwork. Other techniques, such as message identifiers, may be used todetermine if the client resides on a wireless network.

The receiver of the request determines if the requested object resideson another client in the wireless network 1002. For example, servers onthe wireless network may keep service logs containing information aboutprevious requests from other clients, particularly for objects that arefrequently requested. A list of clients that received the requestedobject may be obtained from the service logs. In some aspects, specificclients may be designated as edge servers in the wireless network. Thus,information about which designated edge servers have the requestedobject may be obtained by the receiver of the request. Alternatively,the requesting client may determine if the requested object resides onanother client in the network. A result of step 1002 is theidentification of at least one source client on the wireless networkthat has the requested object or that can easily obtain the requestedobject.

The receiver of the request determines if the requesting client can becommunicatively coupled to at least one of the source clients that havethe requested object 1003. In one aspect, the receiver employs networktopology maps of the wireless network to determine connectivity betweenthe requesting client and the source client(s). In another aspect, thegeographical location of the requesting client, such as GPS dataprovided by the requesting client, is used to select at least one nearbysource client. Step 1003 may be employed as a means for filtering asource client set produced in Step 1002. In some aspects, the Step 1003may be incorporated in Step 1002. In some aspects, the requesting clientprovides its relative network topology to the receiver so the receivercan determine if any nearby nodes have the requested object.

Once a source-client set is identified, the request is routed to atleast one source client in the set 1004, and the source client sends therequested object to the requesting client. Step 1004 may compriseselecting an optimal source client, such as to maximize bit rates,conserve network resources (including client battery life), or optimizeother performance metrics. In some aspects, the requested object may beserved by a plurality of source clients. In some aspects of theinvention, one or more of the steps 1001-1004 may be performed by therequesting client.

With respect to the WWAN configuration depicted in FIG. 5 and the flowdiagrams in FIGS. 2 and 10, a network topology state is generated 201comprising the WWAN client devices 500-505 and 510-513. In one aspect ofthe invention, the network topology state may include WLAN connectionsbetween the client devices 500-505 and 510-513. For example,connectivity information about the individual WLANs 531 and 532 may beincluded in the network topology state, as well as information thatindicates that devices in one of the WLANs (e.g., WLAN 531) maycommunicate with devices in another of the WLANs (e.g., WLAN 532)without loading the WWAN. The network topology state may include deviceswithin PANs, such as devices 551 and 552 in WPAN 533.

The network topology state may include information about connectivity toalternative networks. According to one aspect of the invention, thenetwork topology state includes connectivity information about clients'500 and 501 connections to access point 530. The network topology statemay include connectivity information about the client 512 connection toaccess point 520 via PAN device 551.

Selecting edge server sets 202 may comprise selecting candidate edgeservers that can relay content to WWAN clients while minimizing loads onthe WWAN. In one aspect of the invention, WWAN clients 500, 501, 504,505, and 512 may be selected as candidate edge servers for the set ofclients 500-505 and 510-513. For example, clients 500, 501, and 512 maybe selected because they can submit requests for content via othernetworks that do not load the WWAN. Client 500 may be selected becauseit is a network controller for the WLAN 531, so any content stored onthe client 500 can easily be served to the other clients 501-505 in theWLAN 531. Clients 504 and 505 may be selected since they are WLANclients in both of the WLANs 531 and 532. Thus, content stored on eitherclient 504 and 505 can be made available to all the other clients inboth WLANs 531 and 532.

In some aspects of the invention, selecting edge server sets 202comprises determining which clients a candidate edge server can reachusing peer-to-peer wireless links, such as communications paths spanningcontiguous WLANs connecting the candidate edge server to the clients.

The performance metric for each edge server set 203 may be based onestimated loads on the WWAN and the WLANs 531 and 532, bandwidth andreliability of alternative networks, number of WWAN clients that can beserved, latency, number of hops in peer-to-peer links, and the amount ofdata storage available on each set of edge servers. Calculating theperformance metric 203 may comprise considering how requests for contentcan be routed in the network.

In one aspect of the invention, a WWAN client 502 informs the WLANcontroller 500 that it wants to send a request for content via the WWAN.The WLAN controller 500 typically distributes data to the other WLANclients 501-505 and controls how the WLAN clients 501-505 communicatevia the WWAN. However, the WLAN controller 500 may process the requestby redirecting it to another client in the WLAN 531 that functions as anedge server. For example, if the requested content already resides onclient 504, the content may be served by client 504 directly to client502, or via client 500 to the requesting client 502.

In one aspect of the invention, the request for content is interceptedby the WLAN controller 500 and forwarded to a virtual edge server, whichis a client (such as client 500 or 501) having connectivity to anon-WWAN access point, such as access point 530. The requested contentis received via the access point 530 and routed back to the requestingclient 502 via the WLAN 531.

In some aspects of the invention, a request for content is transmittedvia the WWAN. A WWAN controller, such as in the RNC 561, the SGSN 563,or the GGSN 564, may process the request to determine an alternatedestination for serving the requested content. For example, thealternate destination may be one of the client nodes 500-505 and 510-513selected as an edge server. The request may be forwarded to thealternative destination, or the address of the alternative destinationmay be returned to the requesting node.

In accordance with some aspects of the invention, a content-requestinterceptor resides in the WWAN infrastructure, such as on a clientdevice functioning as a WLAN controller, in the RNC 561, the SGSN 563,or the GGSN 564. The content-request interceptor employs a network mapthat indicates paths to content resources, such as edge servers withinthe WWAN, as well as alternative routes to the Internet that reducecongestion on the WWAN. Such network maps may be employed fordetermining the network topology state 201, selecting candidate edgeservers within the WWAN 202, and/or calculating the performance metrics203. The edge server set having the best estimated performance isselected 204.

In one aspect of the invention, a backpressure routing algorithm isemployed for calculating the performance metrics 203 for each candidateedge server set. Backpressure routing algorithms are well known foroptimizing data flow in peer-to-peer networks. The performance of eachedge server set may be calculated 203 based on how quickly abackpressure routing algorithm reduces queue backlogs. Alternatively,any combination of other performance metrics relating to networkefficiency may be employed. Such backpressure routing algorithms may beemployed for determining an edge server selection in a WWAN and/or theedge servers employed in a conventional CDN.

In one aspect of the invention, step 203 calculates the performancemetric using a backpressure routing algorithm that makes decisions thatgenerally minimize the sum of squares of queue backlogs in the networkfrom one timeslot to the next. Data addressed to node C is denoted ascommodity-C data and stored in a queue denoted as C-queue. In thefollowing aspects, broadcast data may be addressed to multiple nodes,and while it may be stored in a single queue, for the purpose ofbackpressure algorithms, it may be regarded as residing in multiplequeues. The queue backlog of commodity C is simply the amount ofcommodity-C data, which may be measured as the number of packets ornumber of bits. During each time slot, nodes can transmit data to eachother. Data that is transmitted from one node to another node is removedfrom the queue of the first node and added to the queue of the second.Data that is transmitted to its destination is removed from the network.

According to one aspect of the invention, data transmissions for apredetermined number of time slots are simulated, and then theperformance metric is based on the resulting reduction in queuebacklogs. In another aspect of the invention, the number of time slotsrequired to achieve a predetermined reduction in queue backlogs isestimated, and that number of time slots is employed in the performancemetric.

In the backpressure routing algorithm, a transmission rate matrix hasvalues μ_(ab)(t) indicating the transmission rate used by the networkover link (a, b) in time slot t, representing the amount of data it cantransfer from node a to node b in the current slot. This can betime-varying due to time-varying channels and node mobility. There aremultiple transmission matrices μ_(ab)(t) that can be used, since it'stypically not possible for all the nodes to transmit and receivesimultaneously in a given time slot, such as may be due to limitedchannel resources and multiple-access interference.

Let S(t) represent the topology state of the network, which capturesproperties of the network on slot t that affect transmission. LetΓ_(S)(_(t)) represent the set of transmission rate matrix optionsavailable under topology state S(t). In each time slot t, the networkcontroller observes S(t) and chooses transmission rates (μ_(ab)(t))within the set Γ_(S(t)).

In conventional backpressure routing, each node a observes its own queuebacklogs and the backlogs in its current neighbors. A current neighborof node a is a node b such that it is possible to choose a non-zerotransmission rate μ_(ab)(t) on the current slot. It is typical for thenumber of neighbors to be much less than the total number of nodes inthe network. The set of neighbors of a given node determine the set ofoutgoing links it can possibly use for transmission on the current slot.

Typically, the optimal commodity is chosen for transmission between twonodes, wherein the optimal commodity is the commodity that has themaximum differential backlog between node a and node b. For example, foroutgoing link (a,b), a large backlog in node a and a small backlog innode b constitutes a large differential backlog between a and b. Aweight W_(ab)(t) for each link is computed that represents thedifferential backlog. For example, W_(ab)(t) may be proportional to thedifferential backlog. Next, the best transmission matrix for the optimalcommodity is chosen such that the product of the weight matrix and thetransmission matrix is maximized. Routing variable for each transmissionlink are selected based on the selected transmission rates (μ_(ab)(t)).The total of the final weights W_(ab)(t) may be employed in theperformance metric calculation 203.

In one aspect of the invention, commodity data backlogs are based onstatistical models, such as may be determined from client-deviceinformation, such as the type and amount of data each device is capableof receiving, storage capacity, etc. Statistical models may comprisehistorical data, such as historical network loads, historicaldistributions of nodes in a wireless network, historical channel data(such as channel models), etc. In another aspect of the invention,commodity data backlogs are based on current or near-current backlogsreported by each node.

In one aspect of the invention, each node determines its own set oftransmission rates and routing variables. For example, in a wirelessnetwork, nodes typically communicate via pilot signals, control signals,and/or training sequences from which link-quality information can bedetermined. Signal strength, signal-to-interference levels, bit-errorrate, and/or other signal quality metrics may be measured by each nodefor determining its transmission rates and routing variables. Similarly,nodes may report link-quality information to a network controller, suchas an access point or a cellular base station. Such link-qualityinformation may comprise direct link-quality information, such as arequested data rate, or it may comprise indirect link-qualityinformation, such as a power-control requests and/or acknowledgments.

In one aspect of the invention, node links and their respectivetransmission rates are reported to a central processor, which employscommodity data backlogs for calculating weight matrices and selectingoptimal commodities and transmission matrices. A set of candidate edgeservers is selected. For example, the set of candidate edge servers maycomprise a set of pre-selected edge servers plus a candidate edge serverselected from a set of edge servers, such as denoted in a state of astate-transition (i.e., trellis) diagram shown in FIG. 8.

In the trellis diagram, each node corresponds to a distinct state at agiven time, and each arrow represents a transition to some new state atthe next instant of time. For example, each node 801-809 represents aset of candidate edge servers corresponding to a first calculationinterval S₁. Each node 811-819 represents a set of candidate edgeservers corresponding to a second calculation interval S₂. Each node821-829 represents a set of candidate edge servers at a thirdcalculation interval S₃, and each node 891-899 represents a set ofcandidate edge servers at a K^(th) calculation interval S_(K).

Each arrow represents a state transition from a first edge server set ina first (e.g., k^(th)) state (e.g., state S_(k)) to a second edge serverset in the next (k+1)^(th) state (e.g., state S_(k+1)). By way ofexample, but without limitation, the second edge server set may differfrom the first edge server set by one edge server. In one aspect of theinvention, the second edge server set is produced by appending one edgeserver to the first edge server set. In another aspect of the invention,the second edge server set is produced by removing one of the edgeservers in the first edge server set. In yet another aspect of theinvention, the second edge server set is produced by replacing one ofthe edge servers in the first set.

The possible state transitions depend on the rules differentiating onestate from the next, and the set of available edge servers when a statetransition comprises appending or replacing an edge server. The possiblestate transitions may be further restricted, such as to discardidentical sets.

In one aspect of the invention, nodes 801, 811, 821 . . . 891 correspondto appending a first edge server to the set. Nodes 802, 812, 822 . . .892 correspond to appending a second edge server to the set. Nodes 803,813, 823 . . . 893 correspond to appending a third edge server to theset. Nodes 804, 814, 824 . . . 894 correspond to appending a fourth edgeserver to the set. Nodes 809, 819, 829 . . . 899 correspond to appendingan N^(th) edge server to the set. Considering node 801 in the firststate S₁, there is only one possible edge server set, which is the setcomprising the first edge server. Similarly, node 802 comprises thesecond edge server set, and so on. The possible state transitions fromnode 801 comprise appending the second edge server (node 812), the thirdedge server (node 813), the fourth edge server (node 814), . . . and theN^(th) edge server (node 819). There is no state transition from node801 to node 811, since the first edge server is already part of the set.

For each state transition, there is a branch metric. In accordance withaspects of the invention, the performance metric indicates anyimprovement in network performance minus any costs resulting from thestate transition (i.e., the corresponding change in the set of candidateedge servers). This performance metric may be used as a current branchmetric for a state transition. When a new edge server is appended to anedge server set, this typically changes the set of clients served byeach edge server, which changes network loads, congestion, and networkefficiency. There is also an additional cost associated with appendingand edge server.

There are multiple state transitions that connect to node 812. Forexample, in addition to the state transition from node 801, statetransitions from nodes 803-809 connect to node 812. In accordance withone aspect of the invention, accumulated branch metrics for all thestate transitions to node 812 are compared, and the branch correspondingto the highest accumulated branch metric is retained. Thus, each nodemay comprise a single branch from the previous state. An accumulatedbranch metric is computed by adding a previous accumulated branch metricto a current branch metric. Since the state transition 802 to 811produces the same set as the state transition 801 to 812, one of thesebranches can be discarded.

If the state transition from node 801 has the highest accumulated branchmetric at node 812, then the next possible state transitions comprisenodes 823-829. This appending procedure ends when a predeterminedcriterion is met, such as when a predetermined number of edge servers isreached or when the cost of adding an edge server exceeds the benefit.At this point, the best edge server set (i.e., the best path) isselected. Once the best edge server set is selected, the set isforwarded to the network (e.g., the CDN and/or the WWAN) for assigningedge server functions to the selected nodes.

The algorithm may be repeated and/or modified in an update mode. Forexample, an update mode may comprise updating the network topologystate. Based on changing demand or other variations in the networktopology state, the update mode may comprise determining how muchappending an edge server, removing an edge server, and/or replacing anedge server would improves the network's performance metric.Specifically, the best edge server set may be continuously re-evaluated,and it may be updated when certain criteria are met.

FIG. 9 is a flow diagram illustrating a method for selecting edgeservers in accordance with an aspect of the invention. A first step 901comprises determining a network topology state for the network. Forexample, the network topology state may comprise a map of communicationlinks between the node and some performance measurement, such as bitrate, indicating the quality of each link. The network topology statemay include congestion information, such as queue backlogs in each node,which may be used for performing a backpressure routing algorithm 913.

Step 911 comprises determining available edge servers. Typically,information about the nodes is used to determine available edge servers.For example, a CDN service provider may provide a list of nodesavailable to perform content storage services. In a WWAN, storage spaceon the nodes and each node's access to content delivery resources (suchas alternative networks) may be used to determine which nodes areavailable to perform edge server functions. Available edge servers mayoptionally be determined from the network topology state. For example,network congestion, outages, latency, and link reliability may beemployed for determining candidate edge servers 911. Optionally, atrellis may be constructed that depicts possible edge servercombinations (i.e., edge server sets) as a state-transition diagram.

In step 902, one of a plurality of candidate edge server sets isselected 902 for evaluation. A performance metric is calculated 903 foreach set. For the selected set of candidate edge servers, a routingalgorithm may be employed (such as a backpressure routing algorithm 913)from which the performance metric is calculated 903.

The performance metric may comprise an expression of the overallefficacy of routing data to all the nodes in the network, or at least asubset of the nodes. For example, calculation of the performance metric903 may comprise a measure of latency in delivering data to the nodes,the distribution of network loads, bottlenecks, as well as other factorsthat affect bandwidth efficiency and/or the timely delivery of data tothe nodes. In some aspects, the performance metric comprises a costassociated with an edge server selection. For example, there may be amonetary cost associated with using each edge server, and this cost maybe balanced with improvements in network efficiency. In such cases, amonetary benefit may be attributed to each improvement in networkefficiency. Thus, aspects of the invention may comprise integratingbusiness decisions with technical decisions.

The performance metric may correspond to the branch metric for a givenstate transition, such as depicted in a state-transition diagram. Thus,step 903 may comprise calculating accumulated branch metrics for eachstate, and the optimal incoming path associated with each state may bedetermined. A step of performing traceback 904 uses this information todetermine an optimal path through the trellis for updating the best edgeserver set.

A decision step 905 determines whether or not to output the best edgeserver set to the network so that it may be implemented. The decisionstep 905 may also decide whether to continue evaluating candidate edgeserver sets. In one aspect of the invention, the decision step 905compares the best edge server set for a current state to the best edgeserver set determined in a previous state. If the best edge server setin the current state has a better performance value compared to thepreviously selected best edge server set, control may be returned tostep 902, wherein candidate edge server sets corresponding to the nextstate are selected 902. Alternatively, if the best edge server set fromthe previous state is better than the best edge server set of thecurrent state, then the previous edge server set is deemed to be thebest set and is employed in further decision processing in step 905.

The decision step 905 may compare the best candidate edge server set tothe edge server set employed by the network. If the performance of thebest set exceeds the performance of the employed set by a predeterminedmargin, the best set may be forwarded to the network for implementation906.

In accordance with one aspect of the invention, selecting candidate edgeserver sets 902 begins with a state transition that starts at a noderepresenting the edge server set currently employed by the network.

In accordance with another aspect of the invention, the method depictedin FIG. 9 may be employed in an update mode wherein at least step 902 isrestarted each time the network topology state is updated 901.

For clarity of explanation, the illustrative system and method aspectsis presented as comprising individual functional blocks. The functionsthese blocks represent may be provided through the use of either sharedor dedicated hardware, including, but not limited to, hardware capableof executing software. For example the functions of one or more blocksmay be provided by a single shared processor or multiple processors. Useof the term “processor” should not be construed to refer exclusively tohardware capable of executing software. Illustrative aspects maycomprise microprocessor and/or digital signal processor (DSP) hardware,read-only memory (ROM) for storing software performing the operationsdiscussed below, and random access memory (RAM) for storing results.Very large scale integration (VLSI), field-programmable gate array(FPGA), and application specific integrated circuit (ASIC) hardwareaspects may also be provided.

The methods and systems described herein merely illustrate particularaspects of the invention. It should be appreciated that those skilled inthe art will be able to devise various arrangements, which, although notexplicitly described or shown herein, embody the principles of theinvention and are included within its scope. Furthermore, all examplesand conditional language recited herein are intended to be only forpedagogical purposes to aid the reader in understanding the principlesof the invention. This disclosure and its associated references are tobe construed as being without limitation to such specifically recitedexamples and conditions. Moreover, all statements herein recitingprinciples and aspects of the invention, as well as specific examplesthereof, are intended to encompass both structural and functionalequivalents thereof. Additionally, it is intended that such equivalentsinclude both currently known equivalents as well as equivalentsdeveloped in the future, i.e., any elements developed that perform thesame function, regardless of structure.

1. An apparatus, comprising: a processor; and a non-transitorycomputer-readable memory communicatively coupled to the processor, thememory including a set of instructions stored thereon and executable bythe processor for: constructing a trellis having a number of states atleast equal to a number of edge servers to be included in an edge-serverset, wherein each state comprises a plurality of nodes, each nodecorresponding to one of the plurality of candidate edge servers;employing a trellis-exploration algorithm to select the edge-server set,comprising providing interconnects between each node of a first state toeach of a plurality of nodes in a next state, and for each node in astate, selecting a path corresponding to a best performance metric thatconnects to a node in a previous state, wherein each performance metriccomprises network topology information, the network topology informationcomprising channel measurements from candidate edge servers; anddistributing data signals to the edge-server set, the data signals to betransmitted to client devices being served by the edge-server set. 2.The apparatus of claim 1, wherein each performance metric compriseschannel state information from at least one of edge servers and clientdevices in a wireless network.
 3. The apparatus of claim 1, wherein theset of instructions provide for at least one of adapting a networktopology and allocating channels to serve the client devices.
 4. Theapparatus of claim 1, wherein the network topology information comprisesat least one of estimated bit rates, link-delay information, requestsfor retransmission, a wireless communication channel model, congestion,queue backlogs, and latency.
 5. The apparatus of claim 1, wherein thenetwork topology information comprises a time-average of at least one ofserver loads, client-side demand, queue backlogs, bit rates, channelquality, and geographical distributions of client devices.
 6. Theapparatus of claim 1, wherein the network topology information comprisesat least one of a device network topology for each of a plurality ofdevice types and a demand topology for each of a plurality of contenttypes.
 7. The apparatus of claim 1, wherein the performance metriccomprises monetary costs for employing at least one of the plurality ofcandidate edge servers.
 8. The apparatus of claim 1, wherein the trellisexploration algorithm accounts for allocating channels to at least oneof edge servers and client devices in a wireless network.
 9. Anapparatus, comprising: a processor; and a non-transitorycomputer-readable memory communicatively coupled to the processor, thememory including a set of instructions stored thereon and executable bythe processor for: constructing a trellis having a number of states atleast equal to a number of edge servers to be included in an edge-serverset, wherein each state comprises a plurality of nodes, each nodecorresponding to one of the plurality of candidate edge servers;employing a trellis-exploration algorithm to select the edge-server set,comprising providing interconnects between each node of a first state toeach of a plurality of nodes in a next state, and for each node in astate, selecting a path corresponding to a best performance metric thatconnects to a node in a previous state, wherein each performance metriccomprises a function of benefits and costs associated with at least onecandidate edge server; and distributing data signals to the edge-serverset, the data signals to be transmitted to client devices being servedby the edge-server set.
 10. The apparatus of claim 9, wherein theperformance metric comprises channel state information.
 11. Theapparatus of claim 9, wherein the trellis-exploration algorithm providesfor calculating performance metrics upon adapting at least one of anetwork topology and a network topology state of the edge-server set.12. The apparatus of claim 9, wherein the trellis-exploration algorithmprovides for scheduling channels to serve the client devices.
 13. Theapparatus of claim 9, wherein the edge-server set is selected when apredetermined number of iterations occurs or when a predeterminedperformance criterion is met.
 14. The apparatus of claim 9, wherein theedge-server set is selected based on at least one of appending anddeleting at least one edge server from a previous edge-server set. 15.The apparatus of claim 9, wherein each performance metric is calculatedusing a backpressure routing algorithm.
 16. An apparatus, comprising: aprocessor; and a non-transitory computer-readable memory communicativelycoupled to the processor, the memory including a set of instructionsstored thereon and executable by the processor for: constructing atrellis having a number of states at least equal to a number of edgeservers to be included in an edge-server set, wherein each statecomprises a plurality of nodes, each node corresponding to one of theplurality of candidate edge servers; calculating a fitness functionbased on network performance and cost corresponding to each candidateedge server and the network topology information; employing atrellis-exploration algorithm to select the edge-server set, byidentifying a path through the trellis having optimal path metricsderived from the fitness function, comprising providing interconnectsbetween each node of a first state to each of a plurality of nodes in anext state, and for each node in a state, selecting a path correspondingto a best path metric that connects to a node in a previous state,wherein the best path metric comprises the fitness function; anddistributing data signals to the edge-server set, the data signals to betransmitted to client devices being served by the edge-server set. 17.The apparatus of claim 16, wherein employing the trellis-explorationalgorithm comprises performing multiple iterations through the trellisto refine selection of the edge-server set.
 18. The apparatus of claim16, wherein the network performance comprises at least one of apredicted performance and a measured performance.
 19. The apparatus ofclaim 16, wherein path metrics comprise channel state information. 20.The apparatus of claim 16, wherein employing the trellis-explorationalgorithm comprises adapting at least one of a network topology and anetwork topology state of the edge-server set.
 21. The apparatus ofclaim 16, wherein employing the trellis-exploration algorithm comprisesselecting channels to serve the plurality of client devices.